Imaging devices that are structured light active sensing systems include a transmitter and a receiver configured to transmit and receive patterns corresponding to spatial codes (or “codewords”) to generate a depth map that indicates the distance of one or more objects in a scene from the imaging device. The farther away an object in a scene is from the transmitter and the receiver, the closer a received codeword reflected from the object is from its original position (compared to the transmitted codeword) because a propagation path of the outgoing codeword and the reflected incoming codeword are more parallel. Conversely, the closer the object is to the transmitter and receiver, the farther the received codeword is from its original position in the transmitted codeword. Accordingly, the difference between the position of a received codeword and the corresponding transmitted codeword may be used to determine the depth of an object in a scene. Structured light active sensing systems may use these determined depths to generate a depth map of a scene, which may be a three dimensional representation of the scene. Many applications may benefit from determining a depth map of a scene, including camera quality enhancement and computer vision.
Each codeword may be represented by rows and columns of intensity values. For example, binary spatial codes may use 0s and 1s, corresponding to bright and dark intensity values, to represent a binary pattern. Other spatial codes may use more than two different intensity values. Noise in a received codeword may cause an error in identifying the codeword because the “noisy” codewords won't be recognized as the spatial code that was transmitted. For example, for binary codes, such noise may cause a “1” in the transmitted code to be received as a “0”, or vice versa. This may result in a gap, or hole, in a depth map which uses the erroneous codeword. Depth is not estimated at that point with an unrecognized code, leaving a “hole” in the depth map being generated.
Traditional methods of “filling” holes include estimating the depth at the hole based on neighboring depths, by, for example, by averaging depths around the hole. Such an approach may not provide good results if the hole is at or near a boundary of an object, or at any point where there are variations or discontinuities in depth. Accordingly, there is a need for methods and systems to generate depth when there are holes in the depth map, even at points of variation or discontinuity.